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Objectives: To expand upon an observational study published by the Centers for Disease Control (CDC) 
showing an association between school mask mandates and lower pediatric COVID-19 cases. We examine 
whether this association persists in a larger, nationally representative dataset over a longer period. 
Method: We replicated the CDC study and extended it to more districts and a longer period, employ- 
ing seven times as much data. We examined the relationship between mask mandates and per-capita 
pediatric cases, using multiple regression to control for observed differences. 

Results: We successfully replicated the original result using 565 counties; non-masking counties had 
around 30 additional daily cases per 100,000 children after two weeks of schools reopening. However, 
after nine weeks, cases per 100,000 were 18.3 in counties with mandates compared to 15.8 in those 
without them (p = 0.12). In a larger sample of 1832 counties, between weeks 2 and 9, cases per 100,000 
fell by 38.2 and 37.9 in counties with and without mask requirements, respectively (p = 0.93). 
Conclusions: The association between school mask mandates and cases did not persist in the extended 
sample. Observational studies of interventions are prone to multiple biases and provide insufficient evi- 


dence for recommending mask mandates. 
© 2022 The British Infection Association. Published by Elsevier Ltd. All rights reserved. 


Introduction 


There is ongoing scientific debate around mask requirements in 
schools in the United States and other countries during the COVID- 
19 pandemic. To date, there have been no randomized controlled 
trials of mask requirements in children. All analyses of the ef- 
fectiveness of school mask mandates have relied on observational 
studies. Some of these studies report a negative association be- 
tween mask wearing and case rates, while others fail to identify 
any association. 

Studies of this subject face the challenge of controlling for 
fundamental differences between districts and communities that 
choose to require or not require masking in schools. For example, 
a study released by the U.S. Centers for Disease Control (CDC) from 
Arizona found that mask requirements were associated with a re- 
duction in COVID-19 outbreaks whilst another multi-district U.S. 
study reported lower in-school transmission associated with uni- 
versal masking!:?. It has, however, been difficult to rule out the 
possibility that these associations were a result of confounding 
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variables rather than an effect of the masks themselves, as some 
authors have suggested?'+. 

Importantly, too, the findings of these studies were inconsistent 
with more rigorously designed observational studies?” including 
a study using a regression discontinuity design which reported 
no significant effect of masking on SARS-CoV-2 transmission in 
Catalonian schools®. Furthermore, one randomized study in adults 
found no impact of community cloth masking and only a marginal 
impact of surgical masking, though only in those over 50 years 
of age®. Another randomized study in adults failed to find a 50% 
reduction in acquisition of SARS-CoV-2 by the wearer of surgical 
cloth masks in community settings and was not powered to find 
smaller reductions!”. The scientific literature prior to the COVID-19 
pandemic also found low certainty evidence on the effectiveness of 
masks in various settings”. 

Considering millions of schoolchildren have been required to 
comply with mask mandates since the start of the pandemic and 
may be required to do so again in the future, it is incumbent upon 
society to determine if there is any effectiveness of masking chil- 
dren in schools against COVID-19 or other respiratory illnesses and, 
if so, that the benefits outweigh the harms. 

Our study has two goals: First, we replicate and then extend, 
a highly cited observational study by the CDC on school mask 
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mandates by Budzyn et al.'*. The second goal is to identify pos- 
sible sources of statistical bias in observational studies, particularly 
those that are limited in population size, diversity, and duration. 
This is important because studies identifying rather than rejecting 
an association are more likely to be published!“ and consequently 
inform public health interventions despite a potential lack of true 
efficacy. 

In their study, Budzyn et al. found that, following school open- 
ings in the Summer of 2021, pediatric COVID-19 case rates in- 
creased faster in U.S. counties that did not have mask mandates 
in schools, compared to those that did. While the results do not 
show a causal relationship, the study has been cited by the CDC 
in its policy recommendations and in the news media as evidence 
that mask mandates in schools can lower community spread.! Our 
study replicates the same analysis and then extends it, using a 
larger sample of school districts and a longer study duration. 


Methods 


We follow the methodology from Budzyn et al.!? as closely 
as possible. Data on pediatric and adult COVID-19 case rates, by 
county, were obtained from the October 25 release of the CDC’s 
Restricted Case Dataset. Data on school enrollments and mask poli- 
cies were obtained from the data company MCH, the same source 
used by Budzyn et al. County level demographic variables and 
school district to county mappings were obtained from the USS. 
Census Bureau. 

Following Budzyn et al., counties that met the following crite- 
ria were selected for the analysis: (1) a valid school start date and 
a known mask requirement for at least one school district in the 
county, (2) in districts that have made mask policies available, such 
policies apply uniformly to all students (counties containing dis- 
tricts with conflicting mask policies were discarded), (3) at least 
three weeks of data were available with seven full days of case 
data since the start of the 2021—22 school year (the median school 
start date across districts within a county was used). 

Using September 4, 2021 as the cut-off, as chosen by Budzyn 
et al., these criteria resulted in the inclusion of 565 counties. Using 
the more recent data release of November 30, 2021 resulted in a 
larger sample of 1832 counties. This study uses the smaller sample 
to compare with Budzyn et al, and the larger sample to evaluate 
robustness. 

As in Budzyn et al., we conducted a multiple linear regression 
of per-capita pediatric case rates for these 1832 counties, including 
the following controls: median age, race and ethnicity, and popula- 
tion density; pediatric COVID-19 vaccination rates; adult transmis- 
sion rates in the corresponding county and week; percentage unin- 
sured and percentage living in poverty; social vulnerability index 
(SVI) score; Covid-19 community vulnerability index (CCVI) score; 
and fixed-effects for each week after school opening. 

Demographic variables were obtained from the U.S. Census Bu- 
reau. The SVI score was obtained from the CDC and the CCVI score 
from Surgo Ventures. Community transmission is defined as the 
per-capita rate of adult cases in the county in the corresponding 
week. The pediatric vaccination rate is defined as the fraction of 
children in the county ages 12—18 who received two doses of a 
Covid-19 vaccine. Per-capita pediatric case rates are defined as the 
weekly sum of confirmed cases in the 0—19 age group that was 
reported to the CDC for the corresponding county, divided by the 
county population in that age group. 


1 The CDC cites the Budzyn et al study, among many other observational 
studies, at: https://www.cdc.gov/coronavirus/2019-ncov/science/science-briefs/ 
masking-science-sars-cov2.html. The New York Times cited this study repeatedly, 
using causal language to describe the effectiveness of masks, for example at: https: 
//www.nytimes.com/2021/09/24/health/schools-mask-mandate-outbreaks-cdc.html 
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Fig. 1. Pediatric COVID-19 case rates and school mask requirements: 565 counties. 


Following the empirical analysis, we use statistical methods to 
reconcile the difference between our results and those of Budzyn 
et al., We note various biases that can arise when using observa- 
tional data on jurisdictions that choose whether to adopt partic- 
ular public health recommendations. We explore potential cross- 
sectional and temporal biases that can affect observational studies 
of this nature. 


Results 


We examine the relationship between school mask require- 
ments and pediatric cases of COVID-19, replicating the analysis of 
methods of Budzyn et al., as closely as possible. We identified 565 
counties that met the criteria, compared to 520 counties in the 
original study. The discrepancy occurred for at least two reasons. 
First, MCH obtains data on school start dates and mask policies 
from phone surveys of school districts and updates this informa- 
tion frequently. This study uses MCH data that were current as 
of October 15, 2021. Budzyn et al., used information from earlier, 
which was likely to have been different as school district informa- 
tion changes regularly. Second, Budzyn et al., selection criteria for 
counties is ambiguous about districts that cross county lines.? 

Fig. 1 displays results from 565 counties that fit the criteria 
used in the original study and analyzes data from three weeks 
prior to schools opening to nine weeks following opening. After 
two weeks of schools being open, which was the endpoint for 
Budzyn et al., the results align with the findings of the original 
study: non-masking counties had, on average, around 30 additional 
daily pediatric cases per 100,000 children, compared to counties 
with mask mandates. 

Thus, during August—September 2021, mask mandates were as- 
sociated with lower pediatric cases of Sars-CoV-2 in districts where 
schools started by August 15. However, Fig. 1 also shows that cases 
quickly declined in later weeks and did so faster in counties with- 
out mask mandates. In fact, the Budzyn et al., study ended at ex- 
actly the peak of school case numbers for this sample of counties. 
By the end of the ninth week after reopening average daily pedi- 
atric case rates in counties without mask mandates were 15.8 per 
100,000 while counties with mandates averaged 18.3 per 100,000; 
the difference was not statistically significant (p = 0.12). 

Fig. 2 shows a larger sample of 1832 counties, including coun- 
ties which were not included in Budzyn et al., analysis due to later 
school district start dates. Note that there are 3142 counties in the 
United States, but many cannot be classified with regard to school 


2 We attempted to resolve the discrepancy by contacting the lead author of that 
study, but they did not provide assistance and declined to share the code used to 
construct their sample. 
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Fig. 2. Pediatric COVID-19 case rates and school mask requirements: 1832 counties. 


mask policies, due to either insufficient data from MCH or the spe- 
cific criteria used by Budzyn et al. 

The larger sample of counties enables us to employ a much 
broader geographical representation of districts, in addition to in- 
creasing the population size of the study. This is important for 
full geographic representation as southern states are more likely to 
have early start dates. The figure shows the lack of a clear relation- 
ship between mask mandates and pediatric cases. Counties that 
required masks in schools saw slightly larger increases in cases in 
the weeks immediately before and after school opening, but by the 
second week after the start of school there was no statistical differ- 
ence. At the end of week 9, cases per 100,000 capita were 21.8 in 
counties without mask mandates and 24.3 in counties with man- 
dates; the difference was not statistically significant (p = 0.057). 

Fig. 2 also shows that the change in case rates, across various 
weeks, was virtually identical for counties with and without school 
mask mandates. From three weeks prior to the start of school to 
two weeks after school opening, average daily cases rose by 39.0 
per 100,000 in counties without mask requirements, and by 40.2 
per 100,000 in counties with mask mandates. The difference was 
not statistically significant (p = 0.74). Similarly, between weeks 2 
and 9, cases per 100,000 fell by 38.2 and 37.9 in counties with and 
without mask requirements, respectively. This difference was also 
not statistically significant (p = 0.93). 

Next, we estimated a multiple regression of pediatric case rates 
controlling for observable differences across counties (Table 1). The 
first column of that Table uses the same specification as Budzyn 
et al. and confirms the lack of association between school mask 
mandates and case rates: after accounting for covariates, pediatric 
cases in counties with mask mandates were slightly higher, by 1.27 
weekly cases per 100,000, though the effect was not statistically 
significant (p = 0.058). 

In column 2 of Table 1, we show the effect of removing the 
COVID-19 community vulnerability index (CCVI). The CCVI is de- 
rived from, and is highly correlated with, the social vulnerability 
index. Omitting this variable results in a significant positive asso- 
ciation (p < 0.0001) between school mask requirements and pe- 
diatric case rates (see Supplementary Appendix for additional de- 
tails). 


Discussion 


The study by Budzyn et al. examined 520 U.S. counties and 
found that counties without mask requirements in school expe- 
rienced larger increases in pediatric COVID-19 case rates follow- 
ing the start of school compared with those that had school mask 
requirements. Specifically, the increase in pediatric case rates per 
100,000 children starting from one week prior to school opening 
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Table 1 
Regression of pediatric COVID-19 case rates per 100 K. 


From Budzyn From Budzyn et al 


et al (1) minus CCVI (2) 
School Mask Requirement 1.279 3.507 
(0.058) 0.000 
Adult Cases per 100K 1.194 1.196 
(0.000) 0.000 
Percent Uninsured -0.558 -0.353 
(0.000) 0.000 
Percent in Poverty 0.531 0.312 
(0.000) 0.000 
Population Density -0.001 0.001 
(0.162) 0.329 
Social Vulnerability Index -13.558 8.911 
(0.000) 0.000 
Community Vulnerability Index 27.154 
(0.000) 
Percent Non-Hispanic White -0.892 -1.836 
(0.667) 0.380 
Median Age 0.768 0.738 
(0.000) 0.000 
Pediatric Vaccination Rate 1.320 -1.352 
(0.546) 0.540 
Intercept -40.113 -35.973 
(0.000) 0.000 
R? 0.673 0.666 
Observations 12824 12824 


Notes: Values in the table are coefficient estimates obtained from a regression of 
county level reported per-capita pediatric cases of Sars-Cov-2 on the control vari- 
ables listed in the table. p-values are in parentheses. Regressions also include week 
fixed effects, that are not reported. The specification in Column 1 follows Budzyn 
et al. The specification in Column2 omits the CCVI index due to its high correla- 
tion with the Social Vulnerability Index. See text for sample construction and other 
details. 


until one week afterwards was 16.32 for the former group and 
34.85 for the latter. 

We successfully replicated the main result of the original study 
by Budzyn et al. Further analysis, however, shows that the results 
do not hold in a larger sample of school districts, or even in the 
original sample of districts when extended to a longer time inter- 
val. Specifically, when we extend the sample to districts that began 
later in Autumn 2021, which encompasses a much larger portion 
of the country, we find no association between mask requirements 
and pediatric cases. 

Thus, using the same methods and sample construction crite- 
ria as Budzyn et al., but a larger sample size and expanded time 
frame for analysis, we fail to detect a significant association be- 
tween school mask mandates and pediatric COVID-19 cases. The 
discrepancy between our findings and those of Budzyn et al. is 
likely attributable to the inclusion of more counties, a larger geo- 
graphic area and extension of the study over a longer time period. 
By ending the analysis on September 4, 2021, Budzyn et al. ex- 
cluded counties with a median school start date later than August 
14, 2021. According to the MCH data, this heavily over-samples 
regions that open schools by mid-August including Florida, Geor- 
gia, Kentucky and other southern states. The original study would 
not have incorporated data from New York, Massachusetts, Penn- 
sylvania, and other states that typically start schools in Septem- 
ber. While this does not necessarily bias the results, it calls into 
question whether the results of that study can be representative of 
the entire country and suggests at least one important geographic 
confounding variable affects observational studies of school-based 
mask mandates in the United States. 

If mask wearing were assigned or promoted at random, as, for 
example, in the Bangladesh cluster-randomized trial by Abaluck 
et al.°, then the relationship between these mandates and case 
outcomes may have a causal interpretation'’. However, school 
mask mandates in the United States are not only non-random, but 
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likely to be correlated with unobserved factors in systematic ways, 
making it inappropriate to infer causality. 

First, school districts that mandate masks are likely to invest in 
other measures to mitigate transmission and may differ by testing 
rates and practices. Second, the choices made by school districts 
reflect the attitudes and behavior of their community. Communi- 
ties that are concerned about the spread of SARS-CoV-2 are also 
likely to implement other measures, even outside of schools, that 
may eventually result in lower spread in the community and in- 
cluding within schools. Finally, the timing of public health inter- 
ventions is likely to be correlated with that of private behavioral 
changes. Public health measures are typically introduced when 
case counts are high, which is precisely when community mem- 
bers are likely to react to media coverage and change their own 
behavior. 

Our study also uses observational data and does not provide 
causal estimates either. However, there is an important difference: 
while the presence of correlation does not imply causality, the ab- 
sence of correlation can suggest causality is unlikely, especially if 
the direction of bias can be reasonably anticipated'®. 

In the case of school mask mandates, the direction of bias can 
be anticipated quite well. Past research suggests that behavioral 
changes are likely to be positively correlated with public health 
measures to reduce cases, both cross-sectionally and temporally”. 
If so, the bias in the estimated coefficient from a naive regression 
of case outcomes on public health measures will be negative. In 
other words, an analysis that omits the extent and timing of pri- 
vate behavioral changes, even one that controls for covariates, will 
tend to overstate the effect of mask mandates, and run the risk 
of a spurious correlation between mask mandates and declines in 
case rates. 

To see this, consider estimating the following linear regression: 


Yit = bo + biXie + ballit + eit 

where yj, denotes case rates in community i at time t, x denotes 
observed public-health measures such as mask mandates, and u 
denotes variables that can affect case rates, but are unobserved or 
difficult to measure, such as private behavioral changes in the pop- 
ulation. If we expect that x and u have a correlation ô > 0, but 
that u is negatively correlated with y, the bias in the estimated 
coefficient from a naive regression of case outcomes on observed 
public-health measures is b} which is negative'!®. 

In principle, it is possible for the bias to operate in the other 
direction. One mechanism for this would be if districts that re- 
quire masks in schools implement fewer other mitigations, per- 
haps because they believe that mask mandates are sufficient to 
decrease the risk of infection. However, evidence from some large 
school districts does not support this hypothesis. For example, dur- 
ing the 2021-22 school year, New York City Public schools—by far 
the largest school district in the country—required masks for all 
students from September 2021 until March 2022, both indoors and 
outdoors. During this time, both the school district and the city 
imposed some of the strictest mitigation measures in the coun- 
try. Additional measures in New York schools included daily symp- 
tom screening checks, contact tracing of exposed students in con- 
junction with whole or partial class quarantines, staggered lunch 
and recess times, and an extensive ventilation system with two 
HEPA filters in every classroom.? New York City had mask and vac- 
cine requirements for most places, monetary incentives to encour- 
age adult and child vaccinations and many other interventions.* 


3 https://www.nytimes.com/2021/08/26/nyregion/new-york-city-schools- 
reopening.html 

4 https://www.usnews.com/news/best-states/new-york/articles/2021-12-16/ 
with-omicron-in-full-force-nyc-to-hand-out-masks- tests 
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Regardless of their individual efficacies, these interventions could 
have lowered infection rates in areas with mask mandates, con- 
founding the data in such a way that would give the false appear- 
ance of high mask effectiveness. 

Importantly, too, districts with mask mandates were more likely 
to have higher vaccination rates which, at the time of the Budzyn 
study during the Delta variant wave, might have had more of an 
effect on infection and transmission rates!® than later on, with the 
emergence of the more transmissible and immune-evasive omicron 
variant!, Higher vaccination rates at the time of the initial analy- 
sis by Budzyn et al. could have been conflated with the effect of 
masks. This would further support the hypothesis that the corre- 
lation between school mask mandates and other interventions is 
likely to create a spurious negative, rather than positive, bias in 
the estimated effect of mask policies on case rates, as explained 
above. 

An additional possible confounder in our study may be 
differences in testing rates across counties with and without 
school mask requirements, especially as research suggests that 
Republican-led states have historically had lower testing rates?°. 
This limitation would have affected the Budzyn et al study as well. 
In the Supplementary appendix, however, we present both cross- 
state and within-state data which suggest that differences in test 
rates do not affect our results. 

We also demonstrated that removing redundant socioeconomic 
data from the regression analysis actually resulted in a significant 
positive association between school mask mandates and COVID- 
19 cases. No causality can be inferred with the increased cases, 
but this demonstrates how controlling multiple times for simi- 
lar variables can alter results and including the CCVI appears to 
strengthen the association identified by Budzyn et al. As shown 
in the Supplementary appendix, removing CCVI from Budzyn et al. 
initial analysis would have rendered their initial finding insignifi- 
cant. Indeed, it is unclear what the purpose was of including CCVI 
in the regression analysis when it contains redundant information 
to the social vulnerability index (SVI) which the authors also con- 
trolled for. 

An important caveat to our study is that it does not ex- 
clude some effectiveness of mask usage or school mask mandates. 
We do, however, show why relying on observational studies can 
be misleading. Such studies will tend to systematically overstate, 
rather than understate, the effectiveness of interventions for two 
reasons: selection bias caused by particular jurisdictions choosing 
to implement interventions; and omitted variable bias, due to such 
jurisdictions also being likely to implement other, unobserved in- 
terventions. 

Furthermore, studies with significant findings are more likely 
to be published than those with non-significant or negative find- 
ings'*, This is particularly important in the context of the current 
pandemic where publishing studies that fit a certain narrative can 
become a self-fulfilling prophecy rather than an unbiased pursuit 
of truth. 

In summary, expanding upon a widely cited CDC study, and em- 
ploying the same methodology but with a larger, more representa- 
tive dataset over a longer time period, we fail to find the same ev- 
idence that school mask mandates are associated with a reduction 
in county pediatric COVID-19 cases. We demonstrate how observa- 
tional studies can be misleading when used to guide public health 


policy. 


Supplementary materials 


Supplementary material associated with this article can be 
found, in the online version, at doi:10.1016/j.jinf.2022.09.019. 
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